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This paper describes a five-stage process for the 
development, implementation, and evaluation of counseling outreach 
programs. State I takes the reader from the formulation of a germinal 
program idea through the procedures of assessing need for the 
program, assessing of agency resources, building a program planning 
team, and conducting a thorough literature search. Stage II 
sequentially outlines how to specify and enumerate behavioral goals 
for the program, how to develop its delivery system, how to design 
evaluation procedures and what to prepare for running the pilot 
program. Stage III emphasizes the importance of effective program 
publicity, the necessity of developing staff training methods, some 
notes on actually conducting the pilot program and ends with a 
discussion of the importance of studying the pilot evaluation results 
before deciding to embark on Stage IV. Stage IV consists of offering 
the program on a regular basis with refinement of training and 
evaluation methodologies. Finally, Stage V includes the periodic 
re-examinatioii of the program in relation to its target, purpose, 
method and changing agency needs. (Author) 
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Abstract 

This paper describes a five stage process for the development, implementa- 
tion, and evaluation of counseling outreach programs. Stage I takes the reader 
from the formulation of a germinal program idea through the procedures of 
assessing need for the program, assessing of agency resources, building a pro- 
gram planning team, and conducting a thorough literature search. Stage II 
sequentially outlines how to specify and enumerate behavioral goals for the 
program, how to develop its delivery system, how to design evaluation procedures 
and what to prepare for running the pilot program. Stage III emphasizes the 
importance of effective program publicity, the necessity of developing staff 
training methods, some notes on actually conducting the pilot program and ends 
with a discussion of the importance of studying the pilot evaluation results 
before deciding to embark on Stage IV. Stage IV consists of offering the pro- 
gram on a regular basis with refinement of training and evaluation methodologies. 
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Finally, Stage V includes the periodic re-examination of the program in 
relation to its target, purpose, method and changing agency needs. Often 
spin-off ideas are pursued in this stage and re-cycling through the stages 
begins. The five stage model is delineated in sufficient detail to provide 
the reader with a "how to" manual for learning the process. 
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During the past several years a significant role change has been occurring 
in the counseling profession. University counselors have traditionally spent 
most of their professional time assisting individual students with emotional or 
vocational problems. The newer breed of counselor, who used to serve primarily 
those troubled students who found their way to his/her office, has reached out 
to become a growth consultant to the larger academic community. This expanded 
role definition of counseling includes not only remedial help for individual 
students, but a wide range of counseling interventions with administrators, 
faculty, and students. Such counseling interventions have the broader objective 
of changing the living-learning environment in ways that maximize the growth and 
development of all its members. These developnents toward a wider role have 
been seen as well in other student services, and in community agencies. 

The trend began as a strong reaction to college counselors locking them- 
selves into primarily one-counselor to one-client relationships. Getting (1967) 
proposed the model of counseling where developmental tasks were defined as all 
those life experiences that are necessary for intellectual, social, and personal 
growth. Ironically, he challenged college counselors to set for themselves a 
new developmental task. Getting challenged counselors to spend significantly 
more time in identifying the developmental tasks of normal student growth and 
subsequently creating mental health training programs which provide arenas for 
the acquisition of those developmental tasks. 

In 1968, Morrill, Ivey, and Getting designed a counseling center where staff 
members were expected to move outside their safe consulting room and attempt to 
change the institutional environment with the aim of maximizing student growth. 
They named these preventive and developmental helping interventions "developmental 
outreach programs.'* Sensing that they were not alone in their thinking, Morrill 
and Getting (197G) surveyed 397 counseling centers and found that 8G percent of 



the 236 respondents were involved in some type of outreach program, although 
many of these were fairly routine consultation with residence halls, deans, 
etc. However, 122 of the respondent centers were highly involved in some out- 
reach program. 

In a move to facilitate the understanding of outreach programming on the 
part of college counselors, Morrill and Hurst (1971) specified three roles for 
the preventive-developmental counselor. The first role of the preventive-de- 
velopmental counselor is to "contribute to, support, modify, and enhance the 
learning environment." Working to change in loco parentis rules and providing 
faculty consultation to improve teaching are examples of helping interventions 
encompassed by this first role. Second, the college counselor attempts to 
"facilitate maximum utilization of the learning environment by students." In- 
cluded here are helping interventions such as individual and group counseling 
for social/personal skills. The third role of the counselor in an academic 
community is "to study the student and learn his environment and their inter- 
action as a means of providing the necessary data base for the implementation 
of role one and two." 

Our colleagues who predicted the eventual disappearance of individual 
counseling may have been overreacting, but their reaction has yielded an ex- 
panded and potentially more viable definition of the college counselor. The 
most recent conceptualization of outreach programming (Morrill, Getting, and 
Hurst, 1974) reinstates individual counseling to its rightful place, but only 
within a greatly expanded spectrum of counseling interventions. More specif- 
ically, Morrill, Getting, and Hurst (1974) have provided a model which allows 
classification of counseling programs along three dimensions (see Figure 1). 
The three dimensions are: 1) The target of intervention— intervention aimed at the 
individual, his/her primary group, his/her associational groups, or the institution 
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influencing the individuaTs behavior, 2) The purpose of the intervention- - 
intervention of a remedial nature, a preventive nature, or a developmental 
nature, and 3) The method of intervention- -whether the counselor is directly 
involved in initiating the change or indirectly involved through consultation 
and training of others, or the use of media. According to Morrill, Getting, 

m 

and Hurst (1974), all interventions outside of the individual, remedial, and 
^ direct cell are termed "outreach.*' 

What has not yet appeared in the counseling literature is a systematic 
approach to developing outreach programs from the first germinal idea to the 
assimilation of the evaluation data concerning the program's effectiveness. 
Our task in the paper is to outline the five stages we have found to be nec- 
essary in the process of effective program development and discuss the limits 
within which the process must be considered. It is only fair to say that we 
will abstract the process as we have experienced it ourselves in the develop- 
ment of our own outreach programs in the University Counseling Center at 
Colorado State University. We hope to be concrete enough in specifying each 
stage so that our readers, in counseling centers or other campus and community 
agencies, may easily identify where they are in their own outreach programming. 
We also hope that our present conceptualization will assist other counselors 
and human service workers in systematically initiating new outreach programs 
in their own communities. 

After crystal izing a germinal program idea. Stage I consists of demonstrating 
the need, target, and purpose for the program, assessing staff and budgetary re- 
sources, creating a program development team, and conducting a search of the 
relevant practice and research literature. Stage II consists of setting goals 
and behavioral objectives, creating a delivery system for the program, specify- 
ing an evaluation plan and deciding who is going to staff the pilot run of the 
program. Stage IH consists of developing and implementing publicity, intake, 
and staff training procedures, conducting the pilot program with a select 
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population, collecting evaluation data, and deciding whether to extend the pro- 
gram into the next stage. Stage IV consists of offering the program in a con- 
tinuous way to broader populations, refining the training procedures for the 
program leaders, developing supplementary training aids for participants, and 
evaluating program parameters. Stage V involves possible redesign in terms of 
a different target or purpose, or an extension of the original program, and 
systematic reassessment of all aspects of the program. 

Stage I: Idea to Commitment 

Stage I in the life of an emerging outreach program consists of six dis- 
tinct processes. First, a staff member gets excited or strongly interested in 
a germinal idea for a potential program. The remaining five processes may 
occur in almost any order but they must occur before Stage I is completed. 
There must be an agency decision that sufficient agency resources exist to in- 
vestigate the potential of the proposed program. The initiating staff member 
solicits support from interested colleagues and forms a workteam. The initiator 
or workteam must demonstrate need for the program, and a specific target popula- 
tion and purpose of the intervention must be determined. And, finally, the 
workteam searches the relevant practice and research literature for data that 
will support or negate the decision to continue into Stage II. 

If the workteam is formed prior to determination of the target population, 
one or more members of this population should be added later. Likewise, the 
team may decide to add other members with specific relevant skills. 

The Germinal Idea 

How do outreach programs begin? Let us consider some typical sources from 
which the germinal idea for a program originates. Sometimes a counseling center 
staff member says to himself /herself: "I'm interested in working with married stu- 
dents, and they don't seem to have many services available to them. I think I would 
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like to put together a marital enrichment workshop." Another source of germinal 
ideas is the faculty member or administrator who calls the counseling center and 
asks: "Our department doesn't seem to be -doing a very effective job of academic 
advising, could you help us become more effective advisors? Still another source 
of program ideas is the student. For example, a student leader makes this re- 
quest: "We need some leadership training for our student government officers. 
X Could you lead a workshop for us in leadership skills?" Finally, some member 

of the university community may have surveyed a particular student or faculty 
population and found a specific programming need which is referred to the 
counseling center. The possible sources for germinating exciting program ideas 
are as varied as the university community itself. However, there is a critical 
need for additional work in the area of environmental mapping and assessment as 
a part of the determination of needs and priorities for program development. 
Banning (1973) has begun some of this much needed work. 

Demonstrating Need for the Program 

After a germinal program idea is generated, the question, "Just how much 
need is there for the projected program?" must be answered. We feel that it 
is not enough to say, "That group has requested a program, so let's develop 
one." The need must be demonstrated for the projected program in behavioral 
terms. Therefore, one of the initiator's or workteam's first tasks is often 
to systematically survey the appropriate populations within the community to 
assess how much need exists for the projected program (Banning, 1973). Assess- 
ing need in this manner for a projected program also provides valuable informa- 
tion to an agency in setting its program priorities. 

Three distinct questions must be answered in the need-assessment process. 

1. Just how many persons in the target population (students or faculty) 
actually will use the projected program? There are several alternative ap- 
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proaches for determining how many potential users there are for a program. 
The most obvious attack is to ask the population at whom the program is di- 
rected. For example, if the team wishes to initiate a study skills program, 
it might go to the student personnel office and find out how many students 
list "ineffective study skills" as a significant reason for withdrawing from 
school. Or, the team might sample academic advisors to find out how many 
specific requests* in skill improvement are made to them. Another alternative 
is to survey a sample of students actually on academic probation and ask them 
whether they would avail themselves of such a program. Finally, academic 
advisors might be queried as to whether they would actually refer students to 
such a program if an effective one were available. These alternative strage- 
gies may be used with any target population within the university community. 
Seek out a sample of the target population or someone who deals directly with 
that population and ask their reactions to the proposed program. 

2. Are any programs already existing in the university community with 
similar aims? Who offers these programs and how effective are they? The 
answers to these questions, if actively sought, will help in deciding whether 
the proposed program will augment existing services, compete in a non-pro- 
ductive way, or create conflicts with other groups. Accurate information about 
potentially similar programs may lead to a decision to try and join forces with 
other individuals or agencies in creating a proposed program; on the other hand, 
it may underscore the need for the proposed program. 

3. Who specifically is requesting the program? Is it the dean of students 
who has budgetary power over the Center, or is it an angry faculty member who 
has never supported the center and is never likely to do so? This question 
assumes that the counseling center views each new program as a chance to in- 
crease its sphere of influence at all levels of the university community. The 
agency's source of power comes not only from the students it serves but from 
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all those faculty and administrators who understand and support the center's 
operation. An agency may decide that one program has priority over another 
because it has been requested or supported by someone who can effectively 
expand the center's sphere of influence. 

Specification of Target Population and Purpose 

Using the model presented in Figure 1, a specific target population and 
purpose must be determined. This decision is made based primarily on the 
as^pssment of need and the skills and interests of available staff. It is 
also strongly influenced by the targets and purposes already being served 
by the agency. 

Availability of Agency Resources 

A task that must be undertaken is determining whether the agency has staff 
and other resources to begin another outreach program. This is a crucial question 
often overlooked until the consequences of being over-extended become painfully 
evident. Too frequently staff members are given support for developing a new 
program without systematically considering how much actual commitment in terms 
of staff and how much commitment in terms of time is actually involved in pro- 
gram development. A systematic study should be made of all currently existing 
outreach programs in the agency within the context of that agency's philosophical 
priority for program directions. 

More specifically, we suggest first a step-by-step process of systematic 
agency assessment of staff resources. The concepts described in each of the 
following steps will be elaborated later in the body of this paper. First, 
all staff members responsible for currently functioning programs should com- 
plete a checksheet for each program. This checksheet (Figure 2) requires that 
each program director report the person-hours being spent by four categories 
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Figure 2. Checksheet 1— Number of hours being spent by four citegorles 
of personnel for • specific outreich program, by tasks In 
each stage of the program development process. 

Program Name Oate 
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of personnel utilized for each specific task in the program development process. 
Completed, Checksheet I provides the outreach program director with an accurate 
barometer of the progress for his/her program. Second, by compiling the results 
of Checksheet I for all agency programs, it is possible to determine the number 
of agency pe»"son-hours being spent by each kind of personnel in each stage of 
the program development process. Finally, Checksheet II, presented in Figure 3, 
provides a means for mapping all existing programs using the classification grid 
developed by Morrill, Getting, and Hurst (1974). 

These two worksheets assist the staff in answering respectively the follow- 
ing questions: 

1. How much time do we as an agency wish to employ to each stage of 
the program development process, across all programs? 

2. What proportion of each kind of personnel do we wish to employ 
or work toward employing, across all programs? 

3. Which class of program target populations, which type of program 
purpose, and which type of program interventions do we as an agency 
wish to emphasize? 

The authors are not implying that these three questions provide all the 
data needed for an agency to decide which outreach programs to support. Agen- 
cies must also take into account staff interests and skills, agency needs de- 
termined by higher administrators or university expectations, and budgetary 
restrictions. However, when these three questions have been answered by an 
agency, it is a simpler matter to support or reject the initiation of a newly 
proposed program. If the agency, after systematically assessing its program 
goals in employment of staff resources, gives support to a new program, the 
authors feel that the commitment should be at least through Stage III, with 
realistic time support for carrying out all the tasks therein. 
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The Program Development Team 

At some point during Stage I, a workteam is usually formed consisting of 
three to eight persons who are all interested in the program idea. Although the 
main responsibility for a program resides in the initiating agency, team members 
should be solicited from other agencies, academic faculty, and the student body 
because of their interest and skills. 

The utilization of other personnel affords a distinct advantage. We are 
entering the "day of the consumer" in mental health and social services. We 
can no longer get away with planning for , rather than with those who will partake 
of our services. Students from the target population may not only save us from 
embarrassing and time-consuming mistakes as workteam members, but they can also 
be important resources as program implementers. Likewise, professionals in 
allied fields may provide insights and specific skills in the staffing of a 
pilot program. 

We define the four categories of personnel which we recommend utilizing in 
outreach programs as follows. 

1. Professional . This term denotes a professional worker in the field 
in which the program is being developed. In counseling center programs the 
professional is generally a counselor or psychologist. 

2. Allied Professional . This term is applied to the person who is a 
professional in his/her own field, but is engaged in program development or 
implementation in another field. Thus, a minister or academic faculty member 
is an allied professional for counseling center programs. 

3. Paraprofessionals . The paraprofessional is a student or other non- 
professional person given special training by the ayency to perform some of 
the tasks usually performed by professionals. This person may be employed or 
be volunteering his/her services. Some agencies include their graduate student 
trainees in this category; at CSU we do not. 



4. Target Population Member . For purposes of input in Stage I, this is 
a person who is a member of the group or population which is the target of the 
program. 

We have found the team approach to program development to be more effective 
and enjoyable than one or two persons working alone. Besides the obvious benefit 
of additional creativity, specialized skills of individual team members may be 
tapped to facilitate the various tasks in the program development process. 
Perhaps the most special benefit of the team effort is the mutual support and 
encouragement that members can give each other to the sometimes slow and dis- 
appointing parts of the rather long process. Likewise, success often seems 
richer when shared with an interested teammate. Our experience with several 
program development teams has led us to suggest several guidelines for enhancing 
professional effectiveness and socially rich team functioning. 

1. One team member clearly assumes the leadership role of guiding the 
team through the various tasks in the program development process. 

2. Specialization of program development tasks is to be encouraged in 
team members. Team members may even be chosen or invited to participate be- 
cause of special skills they can bring to the process. Someone with evalua- 
tion experfse (research design or statistical skills) is a valuable asset 
to any workteam. A member of the population at which the program is aimed 
can give immediate feedback to proposed ideas or training experiences. An 
individual with conflict resolution skills can help the team through those 
rough spots when differences of opinion effect an impasse in the process. 

3. Whenever possible, specific work responsibilities are assumed by or 
assigned to specific team members with a clear indication of the expected 
completion date. For example, a team member may be asked to provide a first 
draft of ideas brainstormed at a given session so that all may respond to it 

at the next meeting. Another example--a team member assumes the responsibility 
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of compiling a survey questionnaire created by the team to be distributed to 
the target population in assessing need for the program. If specific team 
members do not spontaneously volunteer for the more onerous tasks, then the 
leader must actively solicit a volunteer. 

4. Conflicts and disagreements between team members are best dealt with 
openly as they occur. Such impasses are inevitable and need to be resolved 

% as they happen in order for the team to steadily complete its objectives. We 

have observed that viable outreach programs can be sabotaged because some team 
members are unwilling to try to reach compromise positions or solutions on 
important issues. 

5. Preparing written summaries of each meeting's product of ideas keeps 
all team members involved and informed. Responsibility for recording the pro- 
ceedings of each workteam meeting may be delegated to a regular secretary or 
assumed by various team members at each meeting. 

6. Finally, a regular meeting time and place is clearly specified. 
Equipping the meeting room with newsprint or a blackboard enhances expansion 
of ideas and facilitates brainstorming when it is necessary. 

• 

The Literature Search 

When initiating a new outreach program, a thorough search of the practice 
and research literature affords two direct benefits. Early in Stage I the 
literature search may yield studies that help demonstrate the need for the 
proposed project; other researchers may have surveyed the population(s) at 
which the proposed program is aimed and reported a need for such interventions. 
At the end of Stage I, the literature search may provide ideas that facilitate 
setting behavior objectives and creating training tasks (Stage II) for the 
proposed program. This is especially true when a similar program has been 
attempted and evaluated at another center. 

ERIC 



-12- 



A team approach to completing the literature search, with each member 
sharing sunmaries of important articles and books, saves time. Better yet, 
if graduate assistants or research technicians are available, the workteam's 
responsibility is limited to guiding the search and evaluating the collected 
summaries. Literature searches are often completed by undergraduates taking 
special studies in psychology or education. 

Stage II: Comnitment to Action 
Stage II consists of preparing the new program for a pilot run with the 
target population. First, the broadest goals for the program are delineated 
and translated into behavior objectives. Second, training tasks and the delivery 
system are created to achieve the behavior objectives. Third, an evaluation 
plan is conceptualized and prepared for implementation with the pilot offering 
of the program. And last, personnel are selected and materials developed for 
the pilot program. 

Before actually beginning to generate behavioral objectives, it is help- 
ful to review the classification of the emerging program on two of the three 
dimensions of the Morrill, Getting, and Hurst (1974) program classification 
system. Who is the designated target population? What is the purpose of the 
program intervention? For example, a workshop for improving marital communica- 
tion is classified as follows: Target- primary group; Purpose of Intervention- 
developmental. The team may also have tentative ideas regarding the third 
dimension. Method, at this point, but a final decision should be delayed until 
after behavior objectives are set. By identifying which categories the program 
fits along the two dimensions, the team possesses conceptual guidelines within 
which to delineate the behavior objectives for the program. 

Setting General Goals and Behavior Objectives 

The setting of behavior objectives is perhaps the most important and 
^ least practiced task in the entire program development process. Behavior 

ERIC ^-^ 



-13- 



objective setting is a step-by-step procedure where the broadest goals are 
translated into specific behavioral statements describing the desired effects 
of the program. 

The behavior objective process (Weigel, 1971)'can best be explained by 
walking through the steps with a model program. The program to be discussed 
is a marital enrichment workshop developed at Colorado State University (Hinkle 
\ and Moore> 1971). The workshop is a seven-session, semi-structured marriage 

training experience conducted by supervised allied and paraprofessionals. We 
ask our readers to imagine themselves beginning their program development process 
for this outreach program at Stage II and to observe each step. 

Step I: General Goals . The workteam begins by brainstorming (Osborn, 1963) 
as many general goals for the workshop as possible. From the first rough list, 
the most important general goals are chosen for further specification. For 
example, one such general goal for the marital workshop is "to increase the 
effectiveness of conmuni cation between spouses." 

Step 2: Behavioral Objectives . The workteam next attempts to state each of 
the general goals in behavioral terms. This is done by answering the follow- 
ing questions: "How would program participants behave differently if they 
achieved the general goal?" Therefore, the question to be answered for the 
general goal stated in Step I above would be phrased: "How would participant 
couples conmunicate more effectively with each other as a result of the marital 
enrichment workshop?" One response might be: "Participants would increase the 
number of 'good feedback' statements to each other." There can be several be- 
havior objectives under each general goal. 

Step 3: Observable Behavioral Objective . The next step is to make each be- 
havioral objective as observable as possible. A facilitative question for this 
step is: "How could an observer tell if the program participants actually 
behaved differently as a result of the workshop?" Applied to the above behav- 
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ior objective the question becomes: ^^How could a person observing tell if the parti c 
ipant spouses are making more "good feedback* statements to each other?" The 
answer might be: "They would be making more statements to each other expressing 
specific feelings about specific situations." 

Step 4: Specific Observable Behavioral Objective . Finally, each behavioral 
objective is made even more specific (if possible). There are two helpful ways to 
t increase specificity: 

A. Limit the scope of the behavior . For example, the behavioral objective 
derived in Step 3 above might now be altered to read: "The partici- 
pant partners would be sharing with each other anger felt about a 
particular experience." Obviously, a series of behavior objectives can 
be enumerated for each of several specific feelings about several 
specific situations. 

B. Limit the time, place, person, or context . The behavioral objective of 
sharing anger with the spouse about an experience might be further 
limited to: 1) a time when only the two spouses are present, 2) and 
when each spouse has previously agreed to listen. 

The process of proceeding from general goals to a lengthy list of behavior 
objectives is more difficult to practice than to discuss theoretically. The 
entire task often takes a workteam several hours, spread over several work 
sessions. However, the authors consider the time well spent since the two 
remaining tasks in Stage II cannot be effectively completed without a specified 
list of behavior objectives. The process is often more intuitive and less 
linear than presented above. A program workteam usually wanders back and 
forth between the task of setting objectives and developing implementation 
plans on its way to a finally integrated product. The creative process is 
never as smooth and linear as an analytical description of it. Recognizing 
this fact the workteam moves on its own unique route towards the agreed upon 
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goal~a schematic of general and specific objectives for each unit of the pro- 
gram juxtaposed by the corresponding implementation exercises or training tasks 
designed to meet those goals. See Figure 4 as an example of one such end product 
constructed for one session of an imaginary sexual enrichment workshop for college 
student couples. 

Creating the Delivery System 

In creating the program's delivery system the workteam's first decision is to 
decide on the Method or Methods to be utilized (third dimension on the program 
classification system). Will the program be offered directly by professionals, 
through training of or consultation with ^thers, through media— or through some 
combination of these approaches? Another decision is whether to borrow and modify 
or generate new training tasks from scratch. Often counselors enter the program 
development process at this point without having completed many or all of the 
previous steps; consequently, the choice of whether to borrow or originate the 
training tasks is based more on personal whims than systematic consideration of 
the hard-won behavior objectives. An enumerated list of behavior objectives for 
each of the general program goals greatly sinplifies this decision. 

Let us look at this decision-making process as it occurred with the marital 
enrichment workshop we've been using as an example. First, the summaries of 
related practice literature, already collected, were re-examined in light of 
the workshop's general goals and respective behavior objectives. The question, 
"Does this particular article provide a training task that appears to achieve 
one or more of the behavior objectives for our proposed marriage workshop?" was 
answered. In this manner, a list of such training procedures from other programs 
was compiled to be considered for the delivery system of the workshop. Next, 
the workteam selected from this list the specific training procedures it wished 
to include in some form in the delivery system. The selection included behavior 
goal setting (Wei gel, 1971), the good feedback communication exercise (Bollinger, 
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1969), constructive fighting exercises (Bach, 1969), and nonverbal sensuality 
exercises (Gunther, 1969 and 1971). Of course, all these components were further 
modified to meet the objectives of our particular workshop. Finally, by comparing 
the selected list of potentially effective training elements with the entire set 
of goals and behavior objectives for the program, it became evident which behavior 
objectives would require the development of new training procedures. For our 
marital enrichment workshop, new training tasks had to be created for most of the 
behavior objectives in the area of nonsexual intimacy. 

The workteam's next task is to fully develop all the training procedures 
that comprise the delivery system for the emerging program, including both bor- 
rowed and created procedures. But what concepts or principles guide this pro- 
cess? Think for a minute; what is the broadest goal to be accomplished by the 
training procedures in most outreach programs? Is it not to effect specific 
behavior changes in the target population? The authors have found that a 
behavior change model is most helpful in affording a set of guidelines for de- 
veloping the training tasks for programs which are administered directly to the 
target populations. Keep in mind that this model may not apply to programs 
aimed at creating changes in institutions or to programs of a more indirect 
nature not involving direct participation by target individuals. 

A behavior change training model (Parker, 1971; Ivey, 1971) is presented 
below intwo parts. First, the steps of the behavior change training model 
are explained. Then, these steps are illustrated by walking through a spe- 
cific training procedure from the CSU marital enrichment workshop (Moore and 
Hinkle, 1970). 

The main steps in the behavior change training model for individuals are: 

1. EXPLAIN to the program participants (the target population) the ob- 
jectives of your training procedure, and exactly what the training 
procedure entails. In other words, tell them what you intend to do 

■ and why. 

2. DEMONSTRATE the training procedure for the participants so they can^^^ 
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observe the behavior change objective being reached. This may be 
accomplished by either of two means: a lecture presentation studded 
with clear examples, or an audiovisual model of the behavior change 
objective being effectively achieved. Another way to say this is 
that you "walk them through" the training process to effective skill 
acquisition. 

3. Provide an opportunity for participants to PRACTICE BY ROLE-PLAYING 
the behavior change objective being taught 

and/or 

4. Provide an opportunity for participants to PRACTICE the behavior 
change objective WITH THE ACTUAL PERSONS OR SITUATIONS toward 
which the change is directed. 

In both the "role-playing" and actual practice, FEEDBACK about effec- 
tiveness of PERFORMANCE is given the participants by the program leaders 
and/or other participants. 

5. Conduct a discussion with participants where they compare the train- 
ing objectives with the practicing they have just finished; this 
DISCUSSION allows the participants to INTEGRATE their understand- 
ing of the change objective with their own practice experience. 

The Good Feedback Communication Exercise from sessions two and three of 
the CSU marital enrichment workshop provides a good illustration of the be- 
havior change model in practice. The exercise consists of two elements: 
constructing feedback statements to give to one's spouse, and actually giving 
and receiving the feedback statements. The workshop leader first EXPLAINS 
element one, the criteria for constructing good feedback statements. "Good 
feedback statements are 1) descriptive of feelings rather than evaluative of 
the other person, 2) specific rather than general, and 3) about behavior that 
can be changed, except when giving complimentary feedback." Actual examples 
of good feedback statements accompany the explanation DEMONSTRATING explicitly 
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its meaning, i.e., "I feel angry towards you when you don't pick up your clothes 
in the morning." Then, all participants are asked to translate short descrip- 
tions they've previously written about their partners into good feedback state- 
ments; two complimentary ones, and two negative, angry ones. 

Element two, the three-step process by which the good feedback statements 
are given and received, is then EXPLAINED: 

Step 1: Partner 1 addresses his feedback statement to Partner 2. 

Step 2: Partner 2 says: "What I hear you saying to me is . . . "and 
repeats the statement until Partner 1 indicates that it has 
been received accurately. 

Step 3: Partner 2 response to Partner I's feedback statement with 
"Inside I feel . . . about your statement." 

The three-step process is DEMONSTRATED by an audio tape of a married 
couple actually giving and receiving positive and negative feedback state- 
ments. The demonstration tape offers a model of the communication exercise 
being performed as explained. 

In the next step participants PRACTICE giving and sending their own 
feedback statements as demonstrated, but with an opposite sex partner other 
than their spouses. Participants follow the training model with less anxiety 
and better performance by ROLE PLAYING with a practice partner. FEEDBACK is 
given in the role playing practice by other couples and the workshop leader. 
Subsequently, each participant ACTUALLY PRACTICES the good feedback exercise 
with his/her own spouse. As in the role playing situation, FEEDBACK is given 
by observing couples and the trainer. Finally, all participants share with 
each other their affective and cognitive reactions to all the previous steps 
in the training process. This DISCUSSION facilitates the participants' 
INTEGRATION of the training objectives with their own learning experience. 
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This simple but effective training process aimed as specific behavior 
change is used in the marital enrichment workshop to achieve behavior objec- 
tives for each of the remaining sessions. A summary of the five steps of the 
behavior change training model can be done by using key words in each of the 
steps: Explain, Demonstrate, Role Playing Practice with Feedback, and/or 
Actual Practice with Feedback, and Discussion and Integration. 

>b 

Evaluation Planning 

Besides behavioral izing program objectives, the next least practiced task 
in program development is a design for program evaluation- Why do we counselors 
tend to avoid evaluation like the bubonic plague? Maybe we consider evaluation 
to be the private possession of our "hard-headed" experimental colleagues. More 
likely we are not trained proficiently as researchers and may be generally un- 
interested in questions of experimental design and measurement- For several 
reasons the authors consider the patent omission of systematic program evalua- 
tion a grave error. 

First, objective evaluation data demonstrating the attainment of speci- 
fied program goals substantiates budget requests for programning. Administra- 
tors are going to ask: "What are the aims of that program? How do you know 
that it achieves those aims?" If the team can provide more data than the subjec- 
tive pleasure of leaders and participants the benefits are obvious. 

In the same vein, objective evaluation data demonstrating a program's 
effectiveness serves to increase credibility with the academic community. 
This assumes, of course, that the agency actively informs the larger academic 
community of its programs and research to evaluate those programs. 

Third, and perhaps most important, the evaluation process assists a cen- 
ter in deciding which programs merit continued support. Systematic evalua- 
tion data helps chart the specific directions for improving each program. 
Carefully planned evaluation is one of the primary sources of feedback by 
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which an agency grows. Supporting programs year after year that have not been 
demonstrated to be effective in reaching specific behavior objectives cannot 
be justified. 

The aim of this section is to make the evaluation process less painful 
and more frequently practiced by program developers. The preparation of be- 
havior objectives is itself answering an evaluation question, i.e., "How could 
an observer tell if participants actually behaved differently as a result of 
the program intervention?" Indeed, careful and serious preparation of be- 
havior change objectives is the first step in systematically evaluati ^ i a pro- 
gram. The second step is to secure or create measurement instruments which 
detect specific effects of program interventions. And the third step is to 
design an experiment for assessing the pilot offering of your program. 

• Measurement Instruments ^ Program developers have varied knowledge and 
skills in the area of tests and measurements. If members of the workteu ^ e 
deficit in such skills, then securing an evaluation consultant is strongly 
recommended. The authors have been in the enviable position of working in a 
Counseling Center where a professional has been explicitly hired to assist 
staff members with evaluation concerns. If others are not so fortunate, they 
will need to search elsewhere for an evaluation expert. One pregnant pos- 
sibility is to increase the agency's credibility by asking an academic col- 
league in psychology, education, or statistics to assist in this task. Teams 
may also discover that their computer center employs a staff member specifically 
to help faculty with research designs and measurement. In practice, the team 
may seek assistance from one professional to select or create measurement in- 
struments and from another to help develop a workable research design. 

With or without evaluation consultants, the workteam's rirst evaluation 
decision is whether to use existing personality or behavioral measures or de- 
velop new ones. We have found this task significantly easier when we have pre- 
viously prepared a list of specific behavior objectives for the program. With 
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such a list in hand, a team member may peruse The Seventh Mental Measurement 
Yearbook (1972) and select for further review those tests which appear to 
measure one or more specified behavior objectives. Likewise, a professional (s) 
with expertise in the area of the specific program would know which instruments 
he/she would recommend to measure behavior objectives. Obviously, an enumerated 
list of behavior objectives streamlines this otherwise aimless task. 

If the team has to design its own measures of program effectiveness, the 
list of behavior objectives becomes the best guidelines for facilitating the 
process. If behaviors on the list can be observed directly by independent 
observers, it may be ideal. Usually, this is impossible. Perhaps the sim- 
plest measure derived from a list of behavior objectives is a self-report 
rating scale for program participants on which they rate their own judged 
progress toward each of the specific objectives. Let's look at such a scale 
developed recently at Colorado State University to evaluate a communication 
workshop for married and unmarried couples (Moore and Hinkle, 1970) using 
several items from the Relationship Goals Rating Scale (Uhlemann, 1974), as 
represented in Figure 5. 

Figure 5 : Six selected items from the Relationship Goals Rating Scale (Uhlemann, 
1974), a self report behavior goal rating scale. 

1. Watch and listen to partner's need for physical comforting. 

2. Listen, instead of formulating comebacks, to each other's angry statements. 

3. Share feelings about exciting work experiences. 

4. Ask partner to be comforted when sad or lonely. 

5. State to partner which sexual practices are most satisfying. 

Workshop participants rated each of the items on two separate six-point 
Likert scales: 1) how frequently they act in the manner specified by each item, 
and 2) how important each objective is in their marriage relationship. Of course, 
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the Relationship Goals Rating Scale is fraught with all the shortcomings of a 
self-report ^measure, but it has the advantage of high face validity to program 
participants as a sensible addition to the training process. 

The problem of face validity merits an additional word in regard to eval- 
uating outreach programs. Carefully constructed evaluation plans may go awry 
if they offend participants or detract from the program training process. 
Effective program evaluation requires not only that instruments be reliable 
and valid, but that they make sense to program participants in light of pro- 
gram objectives and content. We have found that most persons will give an 
hour before and after a workshop to take tests providing 1) it is expected 
nonpunitively as part of the program contract, and 2) the tests appear to be 
related to what they actually experience in the program. Explaining to pro- 
gram participants the need for and benefits gained from evaluation, as well 
as giving them feedback of test results whenever possible also softens 
resistance. 

After selecting or creating dependent variable measures, the team 
is ready to design an evaluation plan for the program. The ideal situation 
would provide a true control over the dependent variables to be evaluated. 
In actual practice this is seldom possible. The impossibility of providing 
a true controlled experiment does not diminish the prograrrmer's responsibility 
to evaluate the effects of programs. The use of comparison groups, subjective 
evaluations by participants, process evaluations, change scores, observers' 
reports, and many other devices can be used to provide some feedback about the 
effects of programs. The evaluation of outreach programs requires a flexible, 
creative approach to research. 

Comnon Evaluation Designs . There are three designs we have frequently 
found ourselves employing in evaluating pilot offerings of our programs. Al- 
though this paper is not meant to teach the principles of experimental research. 
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we are outlining the three designs with the hope of facilitating some accultura- 
tion to the evaluation process. We encourage reading of Research Methods in 
Social Relations (Selitz, Johods, Deutsch, and Cook, 1959) for a more thorough 
understanding. 

The design we find ourselves using most often in assessing effectiveness 
of pilot programs consists of pre- and post-program measurements on the de- 
pendent variables with one control group. Figure 5 lists the conditions for 
this simple design. Note that for this design, as in all experimental designs 
involving control groups, the experimental and control groups are randomly 
selected before introducing the treatment (program). 

Figure 5: Conditions comprising the*p^6"P0st design with only one control 
group. 





Conditions 


Experimental 
Groups 


Control 
Groups 


1. 


Prior selection of groups. 


Yes 


Yes 


2. 


Pre-treatment measure. 


Yes 


Yes 


3. 


Random assignment of subjects. 


Yes 


Yes 


4. 


Exposure of Ss to treatment (program) 


Yes 


No 


5. 


Exposure of S^s to uncontrolled events. 


Yes 


Yes 


6. 


Post-treatment measure. 


Yes 


Yes 



Two problems occur when this design is used to evaluate pilot outreach 
programs. First, it is difficult to make subject composition in the experi- 
mental and control groups equivalent. Second, it is difficult to randomly 
assign subjects because of such confounding situations as having to schedule 
experimental group participants when volunteer leaders are free. We have 
attempted to remedy these difficulties by offering a program during two con- 
secutive time periods. Using this design, the program is provided for every 
experimental group. 

34 
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The second design employs program participants as their own control group. 
Two pre-program measures and one post-program are obtained on the dependent 
variable. (Figure 6 lists the conditions for this design.) In this design 
program effectiveness is assessed by comparing the difference between pre-test 

1 and and pre-test 2 and the- difference between pre-test 2 and the post-test. 
Two situations justify the use of this design: 1) when investigators have good 
reason to believe that the pre-program measures will not affect either response 
to the treatment (program) or the post-program measure, and 2) that there are 
not likely to be other influences during the dead period and during the program 
that might bias the subjects • responses to the post-test. When program planners 
can prepare the evaluation instrument far enough in advance, this design is a 
potential alternative. 

Figure 6. Conditions comprising pre-pre-post design with S^s as their own 



control . 

Conditions Subject Group 

1. First pre-treatment measure Yes 

2. Dead period when Ss are exposed to uncontrolled 

events but not treatment (program). Yes 

3. Second pre-treatment measure at the beginning of 

program. Yes 

4. Exposure of S^s to treatment (program intervention). Yes 

5. Post-treatment measure. Yes 



The third design consists of one post-program measure only on the de- 
pendent variables with one control group. Figure 7 lists the conditions for 
this design. 
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Figure 7: Conditions comprising post-test only design with one control group. 

Conditions 

1. Prior selection of groups. 

2. Pre-treatment measure. 

3. Exposure of Ss to treatment (program). 

4. Exposure of S^s to non-controlled events. 

5. Post-treatment measure. 

To maximize this design, subjects should be randomly assigned to the control 
group from a common population prior to introducing the program intervention. 
In reality this is often difficult to do when evaluating an outreach program in 
an applied setting. It is sometimes necessary, although less recomnended, to 
assign subjects on some practical basis or to derive a control group sample after 
or during the program intervention. For example, it may be necessary to treat 
one class and use another as control. If the two groups (experimental and 
control) are similar on general characteristics, including initial position on 
the dependent variable, programmers still will be able to assess the effectiveness 
of the pilot program better than if they had no control group at all. 

Design Efficiency * There are experimental designs more efficient than the 
three above in allowing the team to conclude that thie program causes specific 
behavior changes. Designs employing "lore than one control group which provide 
information about the interaction between important variables are highly valuable 
when practical. Obviously, it is often not practical to invest as much time as 
the more complex designs require in assessing a pilot program. On the other 
hand, the team cannot afford to omit the careful construction of an evaluation 
plan for the pilot program, as the results contribute heavily to the decision 
whether to offer the program again in its first form. As a rule of thumb, we 
encourage program developers to employ the most powerful experimental design 
that their circumstances will afford to evaluate pilot programs. 
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Process Evaluation 

Process evaluation consists of asking participants and program personnel 
(at every level) to give continuous feedback about how they perceive the pro- 
gram's implementation. A very short form which can be used at many junctures 
in the program includes the following: 

1. List several adjectives that describe how you feel or think about program, 
training, etc. 

2. List two or more strengths of program, training, etc. as you experienced 
it. 

3. List two or more weaknesses of program, training, etc. as you experienced 

* 

it. 

4. List two specific ways that program, training, etc. could be improved. 

5. Any additional comments. 

Such a fonn is completed by participants after each or several units 
of the program, by leaders after the same, by leaders after a training session 
for conducting the program, or even after a program workteam meeting. This 
data is continuously monitored and allows immediate changes in the system as 
well as facilitating earned good feelings of competence for a job done as 
intended. 

Evaluation System 

It is vital to develop a workable and detailed system for use of the 
evaluation procedures. This includes not only decisions on design and 
instruments, but also a timetable for evaluation and specific tasks to be 
accomplished at each point. Liaison may have to be developed with the 
computer center or other resources. Various team members will assume re- 
sponsibility for specific tasks in the evaluation process. 
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Preparation for the Pilot Program 

The last task in Stage II is deciding what level of professional training 
is necessary to conduct all or each segment of the pilot program offering and 
developing any materials necessary for the program. 

Staffing . Typically, program developers assume that only trained profes- 
sionals are able to responsibly conduct a new program. In many instances this 
may be true; however, we would ask planners to consider the alternative of em- 
ploying less trained personnel to staff the program whenever possible. This 
may mean a combination of two Methods— direct and training. Such an approach 
may initially be more time consuming, but it has great potential payoff. 

The utilization of other than professional counseling allows more to be 
accomplished and adds insight from a variety of persons. We suggest utilizing 
allied professionals, paraprofessionals , and in some cases, target population 
members in the pilot program. We recommend using only students (paraprofes- 
sionals and target population members) who have been carefully selected and 
trained for a particular program. Student paraprofessionals in CSU programs 
are carefully trained and evaluated prior to working in programs (Delworth, 
Moore, Mil lick, and Leone, 1974). 

We have discovered two principles to be very useful in deciding which level 
of personnel should conduct which training tasks in the pilot program. We hasten 
to add that these two principles also apply to the utilization of personnel 
throughout the entire program development process. 

Principle I : Assign the tasks to be accomplished in a way that best 
utilizes all the personnel involved in the program. We are suggesting with 
this principle that persons involved with preparing or implementing the pro- 
gram have jobs that best fit their other special skills and level of training. 

Principle 2 ; Employ the least trained personnel that will allow the team 
to effectively attain their behavior objectives. Another way to say this is 
that it is recommended that professionals be used only for those tasks that 
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actually demand their expertise. When persons with less training can do the 
job, use them! 

One way to translate these two principles into effective action is to list 
all the tasks that have to be implemented (in this case the training tasks com- 
prising the delivery system of the program) and then decide which ones must be 
carried out by which category of personnel to meet the objectives. With such 
a list the team will find it easier to match the available personnel to the 
appropriate tasks. Of course, other important variables will figure in the 
choice of who conducts the pilot program. For example, the team may have 
designed a sexuality enrichment workshop for married students. Because the 
specific academic community is very conservative, program planners might be 
forced to have the program conducted by a respected professional on the workteam, 
even though an allied professional or paraprofessional could do the job. The 
team would be employing a professional here, not because his/her skills are 
necessarily needed but because he/she lends necessary credibility to the 
endeavor. 

One final consideration may cause planners to violate Principle 2. We 
advise that those professionals who will be trainers of allied professionals, 
paraprofessional s, or other students first actually conduct the program 
tasks that they will be teaching. We recommend this even though less trained 
personnel might adequately do the job. We believe as Robert Carkhuff has 
amply documented (1969) that trainers must be more proficient in the skills 
they teach than those to whom they teach the skills. Actual practice and 
participation in the tasks that comprise those skills is the only means we 
know to gain such proficiency. 

Materials : At this point, all delivery system and evaluation materials 
need to be given final approval by the workteam. In addition, schedules, 
time-line charts, and other supplementary material has to be developed and 
agreed upon by the team. 
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Stage III: Action to Refinement 
Stage III consists of implementing and evaluating the pilot program; 
three clear tasks complement this stage. First, the developed program is 
offered to a selected sample of the target population, with careful monitoring 
by the program development team. Second, both objective and subjective evalua- 
tion data are collected. And third, based on a studied examination of the eval- 
uation data a decision is reached as to whether the program should be continued 
on a regular basis in some form. 

Offering the Pilot Program 

The team has done its homework, demonstrated need for the program, determined 
the agency's ability to support the new program, specified the program's objectives; 
in behavioral terms, constructed an evaluation design for initial assessment of 
the program's effectiveness, and cleverly devised training procedures to achieve 
the behavioral objectives. The workteam has already invested so much professional 
time that the first run of the program seems almost anti-climactic. And yet, 
actually offering the program has its own special excitement. This is the stage 
where you "do your stuff" and when the workteam risks harvesting the fruits of 
all its preparatory labor. 

Publicity and Intake 

In order to enhance achievement of the intended objectives of the previous 
program planning, careful and thorough publicity is vital. We suggest five 
guidelines to assist in publicizing the program. 

Effective advertising: 

1. Is presented well in advance of the program's beginning date 

2. Is readily available to the target population 

3. Is presented in a form palatable to mostjQejnbers^of the target 
population. 
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4. Contains clear statements of the program *s objectives and procedures, 

5. Indicates how interested persons may initiate participation. 

If no one on the workteam is competent to work with publicity, then the 
team should seek extra-team consultation or assistance. Some advertising 
assistance may be secured from the graphic arts services available in most 
universities. Individuals with journalism skills also make good publicity 
consultants. 

Publicity should be based on consideration of reaching the target popula- 
tion, i.e., the persons whom the team wants to participate. 

Training 

Another way to insure success of the pre-program planning is to implement 
continuous training for the new program. If professional personnel are conduct- 
ing the pilot run it becomes a matter of their careful preparation and/or re- 
hearsing before implementing each task in the delivery system. If allied pro- 
fessionals, paraprofessionals, or other students are program leaders in the 
pilot offering, then systematic training of these persons must occur. As a 
general rule, training of program .leaders follows the same behavior change model 
outlined in Stage II for the development of training tasks for program partici- 
pants. A good deal more will be said about training program leaders in our 
discussion of Stave IV. Here it is sufficient to advise program development 
teams to schedule regular meetings for program leaders as a means of continuous 
inservice training. 

Collecting Evaluation Data 

Regularly scheduled inservice training meetings also provide the vehicle 
for gathering and processing subjective evaluation data from program participants 
and leaders. For example, in the marital workshop for student couples (Moore 
and Hinkle, 1970), mentioned earlier in the section on evaluation instruments, 
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program leaders meet weekly with the program development team in order: 

1. To report participants' positive and negative feedback about training 
tasks in each session. 

2. To report their own positive and negative feedback about the training 
process as experienced by them as leaders. 

3. To be trained for the next segment of the workshop. 

It is important that the team systematically solicit positive and negative 
feedback from program participants and leaders, both during and after the 
program. 

The second source of evaluation data comes from participants* pre- and f 
post-responses to the objective measurement instruments. Someone on the program 
development team necessarily assumes responsibility for the pre- and post-adminis 
tration of these evaluation instruments. As we mentioned earlier in the section 
on evaluation planning, it behooves the team to anticipate participant resistance 
to test-taking and assertively deal with it. 

Decision to Continue 

The workteam is now ready to pause and deliberately consider trie future 
of their new outreach program. Subjective feedback from program leaders and 
participants has been compiled in a convenient form for the workteam 's in- 
spection. Likewise statistical analyses on all objective measures have been 
computed and presented in summary form for interpretation. We recomnend that 
the program development team reach an independent decision concerning the 
future of the program before presenting its studied findings to whomever in 
the agency is responsible for coordinating outreach programming. 

A studied decision concerning the future of an outreach program takes 
considerably more work time than is typically given to the process. Several 
hours are usually required for a workteam to ponder the subjective and objective 



evaluation data in light of all the appropriate behavior objectives. Deliberate 
account taking of all available data juxtaposed against the program's specified 
objectives will result in one of three possible decisions: 

1. Continuing the program in approximately the same form because both 
subjective and experimental data are essentially positive. 

2. Continuing the program with modification of a) specific parts of the 
delivery system, b) specific behavior objectives that now seem less meaningful, 
or c) evaluation measures that don't appear to be tapping the desired effects 
of the program. 

3. Discontinuing the program because both subjective and experimental 
data are essentially negative. 

A further word about the modification of objective instruments is necessary. 
When behavior objectives are painstakingly specified before choosing evaluation 
instruments, the chances are greater that those instruments actually measure 
the desired effects of the program. However, this practice does not guarantee that 
the workteam has selected or created a valid instrument. If the team has strong 
evidence or reason to believe that changes occurred in participants because 
of the program intervention, but those changes are not reflected in the 
objective measures used in your pilot study, then it is prudent to take a 
closer look at the instruments and attempt to determine what happened. Valid- 
ation studies may be necessary before deciding whether to use the measure 
again. At any rate, the workteam's evaluation expert or consultant is inval- 
uable in helping make sense of such discrepancies. 

Finally, the workteam's decision about the future of its program, including 
all supporting data, is presented to the person or persons in the center re- 
sponsible for monitoring the allocation of staff resources to outreach program 
development. The program needs to be considered in the context of an up-to- 
date survey of all outreach programs being conducted by the center. As indicated 
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before, the information compiled from the worksheets (Figure 2 and 3) is de- 
signed to help the agency, as a whole, answer three questions: 

1. How much time do we as an agency wish to employ to each stage of the 
program development process, across all programs? 

2. What proportion of each kind of personnel do we wish to employ or 
work toward employing, across all programs? 

3. Which category of target populations, which type of program purpose, 
and which type of program interventions do we as an agency wish to emphasize? 

The program development team's proposed decision about its new program 
comprises a significant piece of data to be utilized in the agency's ongoing 
process of answering for itself the three questions above. Finally, if the 
program development team's decision is to continue the new program in some 
form, and the outreach coordinator concurs, then we close this section of the 
paper with one caution. Be certain to include sufficient work time for all 
personnel involved to effectively complete the tasks in Stage IV of the pro- 
gram development process. 

Stage IV: Refinement to Maintenance 
Stage IV involved the full implementation of the program. Four specific 
tasks are presented in this stage. First, the program is offered on a regular 
basis to larger samples of the target population. Second, effective training 
procedures for program leaders are further developed. Third, materials are de- 
veloped or redeveloped. Fourth, the evaluation process is continued and refined 
to answer more specific questions. 

Offering of the program 

Stage IV still involves working with the original target population, but 
in greater numbers because the team is not ready yet to extend the program to 
other populations. For example, we would not take the marital enrichment work- 
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shop for relatively healthy couples and offer it to couples seeking marriage 
counseling because of severe relationship problems. Continued offering of 
the program allows tightening up or re-doing organizational procedures such 
as publicity, scheduling rooms, collecting evaluation data, or distributing 
training materials. Additional personnel may also be brought aboard to improve 
efficiency in the program's day-to-day operations. 

Refining Training Procedures for Leaders 

Whereas in Stage III training might receive less attention because pro- 
fessional staff acted as leaders, in Stage IV training must receive more careful 
attention. Focus should be on who needs to be trained and how to train all 
personnel most effectively. The amount and type of training that program 
leaders will need depends mainly on two variables: 1) the training experience 
that prospective leaders already possess, and 2) the depth and variety of 
leader skills demanded by the program. 

Selection is an important issue in the first variable. Whenever possible 
it makes sense to select prospective leaders whose background and present 
skills most closely approximate those needed for the program. This cuts 
down training time and effort on the part of both trainer and trainee;. However, 
sometimes other considerations may favor bringing less qualified persons into 
the program as leaders. For example, although the team has a sufficient number 
of experienced allied professionals to serve as leaders, they may decide that 
they want students involved in this capacity and are willing to go to the extra 
effort to train them. Of course, there are always times, often at the beginning 
of the program, when no trained personnel are volunteering. A good rule in this 
instance is: Take the best prospects and plan training from there. 

Training will also depend on the skills required to conduct the program. 
If the leaders need to be group facilitators, consultants, organizers— that is, 
serve in a number of roles, training will need to be more intense and extended. 
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One solution to this problem is to train different personnel for different roles. 

The workteam can also make good use of inservice training to teach and 
improve leader skills. Not all training must occur before the leader is able 
to offer some service. In fact, the leader's confidence and motivation for 
further training may be enhanced by giving him/her a chance to participate in 
a leadership function. On the other hand, the consumer of the program has the 
right to expect a leader who knows what he/she is doing—so the team must be 
careful not to value the growth of the prospective leader over that of the 
program participant. 

If students are being trained as leaders of programs, it is often possible 
to set up a training process which extends over a quarter or semester, and to 
offer academic credit to such trainees. Many departments have special studies 
courses which can be utilized for this purpose. 

Training for leaders will generally follow the methodology outlined in 
Stage II for program participants: that is, explain, demonstrate, role play 
with feedback and/or actual practice with feedback, and discussion to integrate 
the entire process. Trainees may need to do extensive role-playing before they 
work with a "real" population. Some observation of experienced leaders working 
in the program can be especially helpful when interspersed with role-playing, 
if that is not disruptive to the program. We would usually expect the leader 
trainee to achieve a higher level of ability to utilize the skill learned than 
we expect from the program participant. Also, more "cognitions" are built into 
leader training because leaders need and want to understand more of the theory 
and basis of the program than does the average participant. 

An "internship" period, during which the leader trainee is given more in- 
tensive supervision and feedback as he/she works in the program, is a good idea. 
This may be most helpful in programs which are less structured and call upon 
the leader to "fill in the gaps" during implementation of the program. 
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Another important issue is raised by the question "who should train?" 
Often, the original workteam or one professional member of that team does all 
the training. That will work in the beginning, but as the program expands 
this could become very difficult in terms of the professional's time. There- 
fore, we suggest the tasks to be accomplished in training be broken down and 
some of the allied professionals and paraprofessionals be prepared to teach 
parts of the process. This saves professional time, and perhaps more impor- 
tantly, allows a real "grass roots" influence in the training portion of the 
program. Another bonus is that trainers who are concurrently working as 
leaders in the program very often bring a realistic down-to-earth approach 
which the theorizer may have lost. 

A mature training program, then, includes a systematic selection process, 
a well-ordered, sequenced pre-program training experience based on the behavior 
change model, an ongoing inservice training package, and the specification of 
training tasks to be mastered by each level of personnel involved in the pro- 
gram. Each of these training elements deserve individual attention by the pro- 
gram development team. 

Development of Materials 

The team will have developed some rudimentary training manuals for pro- 
gram leaders and/or participants before reaching Stage IV. They now have the 
time perspective and evaluation data to review all aspects of the program and 
decide where written and audio or visual material may further enhance the pro- 
gram objectives. We have found several questions helpful in this regard. 

1. Are we taking time to saj^ things which could just as easily be read 
and understood by program leaders and/or participants? 

2. Are there informational elements that program leaders and/or parti- 
cipants should have available for reference throughout their training or pro- 
gram experience? 
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3. Are there concepts or expected behaviors, which are difficult to get 
across to program leaders and/or participants, that might be more effectively 
presented by some sort of audio or visual model? 

4. Are all training materials for both leaders and participants written 
in a clear, concise style so that the respective leaders can easily implement 
the instructions? 

Continuation of Evaluation Process 

Research is conducted in Stage IV in order 1) to demonstrate the overall 
effectiveness of an outreach program, and 2) to isolate the differential con- 
tributions of various aspects of the program. Evaluation data resulting from 
such research provides feedback for further planning and program refinement. 

There are a number of reasons why it is important to reassess the impact 
of a program. Certainly any redevelopment which occurred as a result of the 
initial evaluation in Stage III should be assessed. Even if no changes have 
been made, it may be desirable to reassess overall program effectiveness 
employing a more powerful experimental design with the larger target population 
samples. Another reason is that when an outreach program is first initiated, 
leaders and participants may be so involved and excited with its "newness" 
that measures of effectiveness can be spuriously evaluated by this "primacy" 
effect. 

The largest bulk of research conducted in Stage IV deals with the various 
parameters of the program that differentially contribute to a program's overall 
effectiveness. This research helps isolate the most and least effective elements 
in the program's delivery system. It is also possible to experimentally study 
the interactions between various participant characteristics, leader character- 
istics, and criterion measure of program effectiveness. For example, a Stage IV 



4o 



-38- 

« 

experiment might be to study the differential effectiveness of each category 
of personnel (professional, allied professional, paraprofessional , and student 
volunteer) as program leaders. 

The evaluation questions raised in Stage IV require the most powerful and 
efficient research design the program situation will allow. Although it is 
* beyond the scope of this paper to discuss the various designs available, designs 

that use more than one control group and provide information about the inter- 
action between important variables should be considered where feasible. 

Finally, it should be emphasized that evaluation data are of little value 
unless they are available and used by those involved. All workteam members and 
program leaders should be kept thoroughly infonmed of all evaluation results, 
for evaluation is the basis upon which changes in the program delivery system 
are systematically implemented. At times it also makes sense to share evaluation 
results with program participants. Pre- and post-test scores can be helpful to 
a participant as he/she examines his/her own goals for participation in the pro- 
gram and individual progress. 

Other important consumfcr gr^^ps of evaluation data are key administrators 
of the program directors. These key persons include the administrators in the 
counseling center, administrators in similar overlapping agencies, and adminis- 
trators who have responsibility for the university division within which the 
program team is working (e.g., the dean of students and selected vice-presidents). 
Failure to communicate with any or all of these individuals may result in lack of 
continued financial support for a program. It is prudent, too, to actually share 
the evaluation process with administrators by actively involving them to some de- 
gree in the interpretation of results. Their commitment to the program is often 
strengthened and, as a bonus, they very often have excellent ideas and suggestions 
for future program development and evaluation. As a shared process with all con- 
cerned, evaluation becomes a vital and exciting part of the program development 
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Stage V: Maintenance to New Directions or Improvements 
Stage V consists of pursuing either of two recycling processes. Team 
members may wish to develop a new program idea which is a spin-off of the 
first program, recyling their new adventure through all stages of the model. 
That is, one aspect of a successful program may be expanded to become an 
independent, though related, program. An example of this process can once 
again.be illustrated by the marital enrichment workteam alluded to throughout 
this paper. After creating a general communications workshop aimed at en- 
riching the marriage relationship, the workteam subsequently decided to develop 
a workshop in constructive fighting and enhancing sexuality for married couples. 
In other cases, the original program may be redesigned to serve another target 
group or another purpose. An example would be a workshop designed to help 
individuals develop communication skills which might be redesigned to enable 
a primary group to improve its communication process. Another example would 
be an interpersonal skill group designed for persons with severe problems in 
this area which might be redesigned to aid persons who are functioning adequately 
but who want to further develop interpersonal skills. Although new spin-offs 
are related to the first program, and therefore call on skills, personnel , and 
training materials already in existence, the team should not assume that its 
agency is committed to a new idea just because it supported the original one. 
So the team starts again at Stage I, often adding new members. In this manner, 
program development turns out to be a continuous process with teams that gain 
more confidence and expertise than they possessed the first time around. 

The second kind of recycle consists of periodically looking at all programs 
in an agency in light of several important questions. These questions recycle 
a program back to the questions asked of it in Stages I, II, III, and IV. These 
questions include: 

1. Is there still a demonstrated need for the program? 
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2. Do the current delivery system procedures still accomplish the 
intended objectives for the program? 

3. Have either the target, program personnel, level of training, 

or implementation changed enough so that an objective reevaluation of the 
program is necessary? 

4. Are staff resources and funding still adequate to maintain the program? 

5. Has agency priority for the program changed? 

These questions need to be answered at regular periods and appropriate 
decisions made based on this reevaluation. All staff in an agency ought to be 
included in the consensual prioritizing of programs being offered. 

A FINAL WORD 

We, the authors, would like to share with you, the reader, what writing 
this paper has meant to us. First, we are very pleased with our own growth 
as professionals j as evidenced by what we have learned about the program de- 
velopment process in the past four years. Second, we discovered anew just 
how much our CSU colleagues have contributed to our current understanding of 
outreach programming. We have been continuously stimulated and encouraged to 
write and develop programs that neither of us would have envisioned a few 
years ago. And third, we have become aware that what we "practice" and what 
^ we "preach" about program development are not always congruent. There are 

many exciting and effective outreach programs being conducted by the staff at 
the Colorado State University Counseling Center, ourselves included; but many 
of our programs have not been as systematically developed as our presentation 
in this paper might imply. 

We are aware that two realities affect counseling center function and 
administration. First, the financial "heyday" for public supported higher 
education is past. State legislatures are simply not supporting unlimited 

^ hi 
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growth in their public colleges and universities. In fact, the opposite is 
more typically the rule than the exception. Counseling centers, along with 
academic departments and colleges, are being asked, often required, to sta- 
bilize growth if not to restrict it. Secondly, there appears to be a trend 
for colleges and universities to question carefully the need for and benefits 
of counseling center functions in the larger context of educational goals. 
This trend has resulted in counseling centers becoming increasingly more 
accountable for the services they offer and has forced centers to specify 
more systematic evaluation of all counseling interventions a sine quo non , 
not only in colleges and universities, but in most other human service agencies 
as well. Although the pressure of accountability can be threatening, we feel 
it will make counselors more systematic and creative in their provision of human 
service Interventions. 

We offer the program development process as described herein as a beginning 
attempt to deal with the above two realities. The systematic development, im- 
plementation, and continuous evaluation of counseling and human service programs 
(not excluding one-to-one development counseling) can assist agencies in living 
mr ^e creatively within budgetary limits, as well as help insure the continued 
financial support for effective interventions and programs. 
^ The process outlined in this paper is very behavioral in bias. The reader 

must be cautioned to keep the methods and language of evaluation from becoming 
an end in themselves. Human service programs created for personal growth by 
people are infinitely richer than the behavioral objectives constructed to 
linearly describe and evaluate such programs. Such programs are usually in- 
tuitively conceived in the reverie of a daydream or a very nonanalytical moment. 
Sometimes such noncogniti ve, intuitive sparks give birth to the whole gestalt 
and even see how it will work before a behavior objective is even attempted. 
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Our behavior objectives and evaluative analyses are not the spark that kindled 
the generative flame of ideas, not the rich experience or program participants 
and leaders who partake of it. Our behavioral analyses simply round out and 
complete our consciousness of the program experience so that it may be shared 
and compared in a consensually used scientific language. If we forget this 
fact our programs may become as sterile and lifeless as the statistical analyses 
computed and reported to demonstrate their very joie de vivre . 

Finally, we do not wish to convey that our model of program development is 
a finished product; it certainly is not. We anticipate that as we share with 
our colleagues the process of program development as we currently understand it, 
changes in the model will occur. We would very much appreciate feedback con- 
cerning which aspects of our work are most and least helpful to others in the 
field, and why. 
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